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Abstract 

A set of imaging techniques based on Fuzzy Reasoning 
(FR) approach was built for NASA at Kennedy Space 
Center (KSC) to perform complex real-time visual- 
related safety prototype tasks, such as detection and 
tracking of moving Foreign Objects Debris (FOD) during 
the NASA Space Shuttle liftoff and visual anomaly 
detection on slidewires used in the emergency egress 
system for Space Shuttle at the launch pad. The system 
has also proved its prospective in enhancing X-ray images 
used to screen hard-covered items leading to a better 
visualization. The system capability was used as well 
during the imaging analysis of the Space Shuttle 
Columbia accident. 

These FR-based imaging techniques include novel 
proprietary adaptive image segmentation, image edge 
extraction, and image enhancement. Probabilistic Neural 
Network (PNN) scheme available from NeuroShell™ 
Classifier and optimized via Genetic Algorithm (GA) was 
also used along with this set of novel imaging techniques 
to add powerful learning and image classification 
capabilities. 

Prototype applications built using these techniques have 
received NASA Space Awards, including a Board Action 
Award, and are currently being filed for patents by 
NASA; they are being offered for commercialization 
through the Research Triangle Institute (RTI), an 
internationally recognized corporation in scientific 
research and technology development. Companies from 
different fields, including security, medical, text 
digitalization, and aerospace, are currently in the process 
of licensing these technologies from NASA. 


Real-Time Detection and Tracking of Foreign Object 
Debris (FOD). 

The motion detection is performed generally at the pixel 
level, at the edge level, or even at higher features levels. 
Existing features level algorithms for time-varying outdoor 
background require more computational efforts with respect 
to those working at the pixel level, but they provide more 
accurate information for higher system modules. This new 
motion detection system is not performed at the pixel level, 
provides accurate information, and supersedes the 


robustness of the existing high-features-level algorithms 
requiring less computational time. 

In the detection and tracking of FOD’s during the NASA 
Space Shuttle liftoff, the prototype image analysis system 
uses segmentation via FR techniques to binarize the image 
and generate binary blobs in each image frame. Further 
blob analysis on consecutives image frames allows the 
system to determine which blobs are actually moving 
FOD’s. 

The image background during the liftoff is quite complex as 
moving objects that are not FOD’s (ice flakes, fume clouds, 
external fuel line connections, etc.) constantly appear at the 
scene taped in black and white format. The prototype 
system has shown the capability of not only detecting actual 
FOD’s observed in the tape images but also keeping close 
track of them (see figures 1 and 2). The trajectory of the 
debris is drawn in real time as the FOD is detected. Figure 3 
shows the final trajectory of both detected FOD’s in a bright 
background due to the accumulation of ice flakes (generated 
during the liftoff) on the camera lens. In regard to the 
computational time, the system was also able to process and 
execute motion detection features on every one of the 30 
frames per second generated by the image acquisition 
system. 



Figure 1. Moving debris (apparently a screw) detected and 
its trajectory tracked in real time. 



Figure 2. Second moving debris (unknown) detected and 
its trajectory tracked in real time. 



Figure 3. Trajectory of both moving debris drawn in real 
time and saved for further analysis. 

This real-time object moving detection system is currently in 
its preliminary stage. The system’s concept and its algorithm 
have proven to be robust. Further work on implementation 
and validation is still needed. The current system works 
using black and white image format. Image acquired in color 
would lead to an even more robust system that is not 
necessarily slower because a multiprocessing CPU approach 
is quite feasible. Individual CPUs would independently 
process each primary color. 


The potential applications for object real-time moving 
detection and object’s tracking in complex image background 
include analysis of aerial images, dynamic surveillance, robot 
vision, detection of moving target, analysis of traffic flow, 
and autonomous vehicles. 

Real-Time Visual Anomaly Detection on Slidewires. 

Visual analysis prototype system uses slidewire images 
acquired by a NASA-built image acquisition system known 
as Cable and Line Inspection System (CLIM). The system 
performs image segmentation using the same segmentation 
FR-based techniques utilized in the above FOD detection 
prototype application to separate the slidewire image from 
the image background, preprocesses the image, and performs 
the final segmentation to retrieve regions/blobs with potential 
anomalies or defects. Regions/blobs with anomalies or 
defects are classified and finally retrieved using a learning 
model built via available PNN and GA techniques. In 
experimental tests the visual system detected (in real time) all 
the anomalies classified under the same anomaly class (loose 
slidewire strand) during the training session. 

NASA built CLIM to provide a means of automated image 
acquisition of the fourteen 1 000-foot slidewires used in the 
emergency egress system for the NASA Space Shuttle at 
KSC. CLIM eliminates the hazardous, manpower- intensive, 
and time-consuming methods previously required to 
manually maintain the emergency egress system at peak 
performance. The inspection of the emergency egress system 
using CLIM still requires having a person continuously 
watch cable images for more than 48 hours. 

Figure 4 shows astronauts practicing emergency egress at the 
Shuttle launch pad; Figure 5 shows CLIM being tested at the 
Shuttle pad. 



Figure 4. Astronauts at the emergency egress system 
at the Space Shuttle launch pad. 





Figure 5. CLIM sliding a slidewire and acquiring images 
via three cameras positioned in different angles. 

The intelligent anomaly detection system will be integrated 
to CLIM to completely eliminate the need for human 
intervention in detecting anomalies. The system will not 
only detect the anomaly but also document it. Images 
acquired by CLIM are being used to test the intelligent real- 
time anomaly detection system. Figure 6 is a PC’s screen 
snapshot showing a slidewire anomaly detected by the 
system. 


Figure 6. System software screen displaying a detected 
anomaly (a loose strand). 
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The potential applications of this anomaly detection system 
in an open environment are quite wide because there is no 
need for special lighting. One immediate application at KSC 
is the detection of anomalies of the NASA Space Shuttle 
Orbiter's radiator panels. Potential applications include 
automated visual inspection on spacecraft, ground, and flight 
equipment. A commercial application using the imaging 
techniques and entire learning approach is currently being 


explored to build a visual ovarian tumor detection system via 
ultrasound images. 

X-ray Image Enhancement. 

The above FR-based technologies have also proved its 
prospective in enhancing x-ray images used to screen hard- 
covered items leading to a better visualization. Since the 
appearance of an x-ray film image is a plane in two 
dimensions of a structure built in three, the image is seen to 
contain a number of regions within the object superimposed 
on the background level. Therefore to extract the contours of 
these different regions, it is necessary to have prior 
enhancement of the contrast levels among them. The FR- 
based enhancement technique developed at NASA 
successfully deals with these problems. 

Preliminary tests to enhance x-ray images acquired at 
Johnson Space Center to screen pyrotechnic devices were 
conducted with very promising results. Figure 7a shows an x- 
ray image of a pyrotechnic valve. Figure 7b shows the FR- 
based enhancement of the original x-ray image (figure 7a) 
leading to an unquestionable better visualization of the 
pyrotechnic devices. Figure 7c shows the FR-based image 
segmentation performed to the original x-ray image (figure 
7a) that leads a perfect binary separation of the devices 
(white) from the rest of the image or background (black). 

Both figures 7b and 7c images were generated using the same 
FR-based techniques. 



Figure 7a. Original x-ray image of two pyrotechnic valves. 



Figure 7b. X-ray image enhancement via edge extraction. 



Figure 7c. X-ray image segmentation via binarization 


The screening enhancement might be easily applied to x-ray 
images currently acquired to screen luggage at airports, 
containers in terminal ports, etc. 





Image Analysis on Columbia Accident Investigation. 

The set of FR-based imaging technologies was utilized in 
support of the Space Shuttle Columbia accident investigation 
on the detection and trajectory-speed estimation of the foam 
debris that hit the Orbiter left wing. Two image acquisition 
sources, running at different frame rates and acquiring 
Shuttle’s images from different angle views and positions, 
were used to automatically detect the foam debris and 
estimate its 3D trajectory as well as its respective 3D speed. 

The detection of the foam debris and determination of its 
respective borders/perimeters at each one of the 2D image 
frames acquired by the two image acquisition sources were 
performed by the FR-based image techniques. The image 
techniques used here were able to autonomously (not 
interactively) detect image blobs from two different image 
acquisition sources, one a film camera with a relative good 
resolution and definition and the other a video camera with a 
poorer resolution and definition. These techniques also have 
proven to be effective in detecting moving objects or debris 
during the Shuttle liftoff. In contrast with other moving 
object detection approaches, these techniques do not need 
consecutive frames to detect moving objects. These 
techniques actually detect the image blob formed by the 
debris. The geometrical properties (center of mass, area, 
perimeter, etc.) of the image blob are also those of the debris. 
The center of mass is the one used to physically locate the 
debris in 2D images. 

The FR-based segmentation technique also used in all above 
applications detected the foam debris in all image frames 
generated by both image acquisition sources (film and 
video). Figure 8 shows two consecutive images acquired by 
the same film-based image acquisition source within a 16- 
millisecond time difference. In both image frames the 
contours of the detected foam debris were automatically 
drawn. Figure 9 shows the third image acquired by the video- 
based image acquisition source 4 ms after the first image was 
acquired by the film-based image acquisition source (see 
figure 8). The foam debris was detected by the same FR- 
based segmentation technique and marked with a yellow dot. 

Additionally, this work presents a simple-to-implement 
approach to automatically determine the position, trajectory, 
and speed of moving debris when images of the objects are 
acquired by 2 image acquisition sources running at different 
frame rates. 

Conclusion. 

One of the main challenges in next-generation image 
processing will be the capability of building autonomous, 
noniterative, flexible, adaptable systems that are capable of 
performing complex image analysis. After three decades of 
research, there is still not a general computer vision approach 
capable of providing the technology to develop a general- 
purpose vision system able to deal with different tasks and 
images (like the human vision apparatus). 



Figure 8. Images acquired from the same image acquisition 
source within 16-ms difference. 



Figure 9. Image acquired from a second image acquisition 
source 4 ms after the first image (figure 8) was 
acquired. 

A set of imaging techniques based on FR approach and 
intended for use in general-purpose computer vision systems 
was built for NASA at KSC to perform a variety of real-time 
image analysis and information extraction from varying 
scenes or images. NASA is currently offering these 
techniques for commercialization. Companies from different 
fields (including security, medical, text digitalization, and 
aerospace) are currently in the process of licensing these 
technologies from NASA. 
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